Hospital quality scores really do reflect patient outcomes

During the heated debates over health care spending, there are often calls to “pay for quality” rather than paying for every service provided. Medicare, for example, is starting down the road of tying hospital quality scores to reimbursements. Some in the health care industry say this is unfair, as a score alone does not account for the many factors that impact patient outcomes.

But a working paper [PDF] co-authored by MIT Sloan professor Joseph Doyle found that such quality scores are meaningful: patients admitted to hospitals that scores well versus one that scores poorly — a “two standard deviation change” — see a 15 percent reduction in hospital readmission and mortality.

“The basic point of the research was to see whether these often-used quality measures are informative or whether they are confounded by differences across hospitals in the types of patients they treat,” said Doyle, an associate professor of management and applied economics and director of the MIT Sloan Initiative for Health Systems Innovation. “We have these existing measures. Are they useful or not? We’re happy to say that they are.”

Scores sometimes maligned. 
In 2002, the Centers for Medicare and Medicaid Services began to require public reporting on patient responses to satisfaction and quality surveys. Results are posted on a website called Hospital Compare. The government’s aim was to improve accountability for hospitals and help patients make more informed choices on where to receive care.

Former president Barack Obama’s health care law took the concept one step further, tying quality scores to Medicare reimbursements. As of 2017, facilities that get low scores on the Hospital Consumer Assessment of Healthcare Providers and Systems survey could face a 2 percent reduction in reimbursements.

The use of quality scores is not without controversy, Doyle said. Hospitals with low scores often claim that they have fewer resources, more complex cases, or less staff than hospitals with high scores. Urban hospitals may have more trauma cases; suburban hospitals may have wealthier, healthier patients.

What’s more, focusing on patient satisfaction may not improve care. A 2012 UC Davis study found that patients who reported the highest satisfaction also had higher hospital admission rates, higher out-of-pocket costs, and a higher mortality rate. Researchers suggested that these patients were more likely to receive duplicative, unnecessary, or even improper care (such as antibiotics to treat a viral infection or higher doses of pain medication than necessary).

Better quality, better outcomes. 
A randomized trial would best determine whether high quality scores beget better outcomes, but patients wouldn’t willingly go to a hospital with low scores just in the name of research.

However, when patients dial 911, the decision about which hospital to visit is partly out of their hands. The nearest ambulance is dispatched, and where patients go often depends on the ambulance company’s preference. Doyle described this ambulance assignment as “effectively random.” Doyle’s paper examines five years of Medicare claims data for patients admitted to the hospital with one of 29 non-discretionary conditions — those that require a 911 call and immediate attention. Patients treated in hospitals with high quality scores — based on Hospital Compare, Hospital Consumer Assessment of Healthcare Providers and Systems, and Centers for Medicare and Medicaid Services data — have a lower likelihood of readmission, as well as a decreased likelihood of dying within one year of admission, than those treated for the same condition in a lower-quality hospital.

“We were trying to compare apples to apples, to see if the report card measures turn out to be informative,” Doyle said. “When we compare very similar patients who go to different hospitals, the report cards hold up: Better quality, better outcomes.”

Builds on previous research. 
The recent working paper builds on Doyle’s previous research on the measurable factors that impact patient outcomes. A paper recently accepted for publication [PDF] in the Journal of Health Economics,  concluded that patients effectively-randomly assigned to hospitals with a high rate of transfers to skilled nursing facilities had poorer health outcomes compared to hospitals that send more patients home instead.

Taken together, the findings of the research suggest that quality scores are an effective tool for helping patients choose where to receive care, Doyle said. “You want a hospital that gets people home and healthy,” he said.